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Abstract 

Neuroscientists have studied the relationship between nerve cell morphology and 
function for over a century. To pursue these studies, they need accurate three- 
dimensional models of nerve cells that facilitate detailed anatomical measurement 
and the identification of internal structures. Although serial transmission electron 
microscopy has been a source of such models since the mid 1960s, model reconstruc- 
tion and analysis remain very time consuming. We have developed a new approach 
to reconstructing and visualizing 3D nerve cell models from serial microscopy. An 
interactive system exploits recent computer graphics and computer vision techniques 
to significantly reduce the time required to build such models. The key ingredients 
of the system are a. digital "blink comparator" for section registration, "snakes," or 
active deformable contours, for semi-automated cell segmentation, and voxel-based 
techniques for 3D reconstruction and visualization of complex cell volumes with in- 
ternal structures. 
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Figure 1: A hippocampal pyramidal cell with the soma and the dendrites (den) 
visible. The axons are from other hippocampal cells that form synapses with these 
dendrites. At higher magnification the dendritic spines are seen protruding from the 
dendrite. Bars - l ()//,m. (Reproduced from (Harris et al., 1980) with permission from 
the publisher.) 

1 Introduction 

Neuroscientists search for links between neuronal dendritic morphology and behavior, 
a.nd morphology and disease by studying the relationship between the morphology and 
function. Detailed morphological studies require accurate three-dimensional models 
of nerve cells that facilitate anatomical measurement and identification of internal 
structures. Neuronal dendrites and their protruding dendritic spines can be seen with 
a light microscope (Figure 1), but the resolution is insufficient for detailed anatomical 
measurement and the internal structures are not visible. To date, the only method 
available for detailed measurement and study of internal cell structure is through 3D 
reconstructions from serial electron microscopy, or serial EM (Figure 2) (Harris and 
Stevens, 1988; Harris and Stevens, 1989; Stevens and Trogadis, 1984; Wilson et al., 
1987). 

Reconstructions from serial EM have been produced almost since the invention of 
the electron microscope. Initially, the reconstructions were purely manual, but over 
the years they have relied increasingly on computers. Even with current computer- 
assisted techniques, the reconstruction of a, 5fim dendritic segment with all of its 
spines and synapses, along with the quantitative a.nalysis of the reconstructed model, 
can take up to three months of work. By contrast, the tissue preparation and EM 
photography take only about two days! It is remarkable that so many neuronal 
reconstructions have been made because "... the incredible investment in time and 
energy necessary to reconstruct cells is nothing short of heroic" (Stevens and Trogadis, 
1984). 
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We present a new interactive approach to reconstructing and visualizing 3D nerve 
cell models from serial microscopy, and describe an interactive system which exploits 
recent computer graphics and computer vision techniques to reduce significantly the 
time required to build such models. After presenting a more detailed review of the rel- 
evant neurophysiological motivation for our work and its relationship to prior efforts, 
we describe the key components of our approach to reconstruction and analysis of neu- 
ronal dendrites. Our prototype system currently features a digital "blink comparator" 
for section registration, "snakes," or active deformable contours, for semi-automated 
cell segmentation, and voxel-based techniques for 3D reconstruction and visualization 
of the 3D morphology of dendrites along with their internal structures. 

1.1 Background 

A nerve cell, or neuron, has four constituent parts: the cell body (soma), the dendrites, 
the axon, and the presynaptic terminal of the axon (Kandel and Schwartz, 1985). 
The soma is the metabolic center of the cell, the dendrites are the receiving units, 
the axon is the conducting unit, and the presynaptic terminals are the transmitting 
units. The areas of contact between the presynaptic axonal terminals of one cell and 
the dendrites of another cell are called the synapses. Most synapses are located at 
the end of protrusions on the dendrites, called the dendritic spines (see Figure 1). 

In humans, the dendritic spines are lost or change shape both with aging (Feldman 
and Dowd, 1975) and with diseases that affect the nervous system, such as dementia 
(Catala et a.l., 1988), brain tumors (Spacek, 1987), Down's syndrome (Marin-Padilla, 
1976), epilepsy (Scheibel, Crandall and Scheibel, 1940), Huntington's disease (Grav- 
eland, Williams and DiFiglia, 1985), and alcoholism (Ferrer et al., 1986). Detailed 
anatomical descriptions of the synapses and dendritic spines will provide new under- 
standing about their function, thus improving opportunities for understanding the 
underlying causes and effects of these diseases. 

The dendritic spine is positioned so that changes in its morphology could modu- 
late the transfer of information from the synapse to the dendrite (Brown et al., 1988a; 
Wickens, 1988). Direct physiological study of the relationship between dendritic spine 
morphology and function has been impossible because of their small size. Several sim- 
ulations with theoretical models have shown, however, that small changes in morphol- 
ogy could change the biophysical properties of the spines (Rail, 1974; Crick, 1982). 
Several laboratory studies have shown that dendritic spines change shape during mat- 
uration, following experience, and in response to direct physiological stimulation of 
the presynaptic axon. Repeated, or "tetanic," stimulation causes an enhanced synap- 
tic efficacy, which is referred to as long-term potentiation (LTP), a leading candidate 
for a synaptic explanation of behavioral learning (Brown et al., 1988b). Anatomical 
analyses of stimulated dendrites have revealed swollen spines and changes in the size 
of synapses (for a review see (Harris, Jensen and Tsao, 1989)). Thus, a change in 
morphology might contribute to, or represent, the increase in synaptic efficacy which 
is observed after stimulation. 

Research has also shown that the cytoskeleton and the organelles internal to the 
neuron, such as neurofilaments, microtubules, mitochondria, and smooth endoplasmic 
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Figure 2: An EM photomicrograph from a, section of a rat hippocampus. The dendrite 
(den) is located in the center, and a large spine is protruding from its right side. The 
mitochondrion (mc), microtubules (mt), some smooth endoplasmic reticulum (ser), 
and the synapse (syn) are indicated. Bar = Ifim. 



reticulum (see Figure 2) may control the shape of the nerve cell (Harris and Stevens, 
1988; Harris and Stevens, 1989). The challenge is to find links between neuronal 
morphology and physiological function in the normal and diseased brain (Brown et al., 
1988a). 



1.2 Previous work 

Registration and Reconstruction: Early EM reconstruction techniques were en- 
tirely manual (Stevens et al., 1980). A photograph from an electron microscope was 
illuminated from below and the outline of the structures of interest were traced on a 
sheet of acetate positioned on top of the print. Next a photograph of an adjoining 
section was illuminated and aligned with the trace from the previous section, and its 
structures were then traced on a new sheet of acetate. This process was repeated 
for all sections. The thickness of the acetate and the magnification were chosen so 
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that a fairly accurate model would result by inserting a number of blank sheets of 
acetate between each trace. Finally, an artist would produce a 2D illustration of the 
3D model. 

Over the years, reconstruction from serial EM has become increasingly computer 
assisted. In a system developed by Stevens et al. (1980; 1984), the EM negatives are 
first rephotographed onto a 35mm filmstrip. Next, the filmstrip is mounted on a film 
transport, which in turn is mounted on a stage driven by two computer-controlled 
stepping motors. The first image is digitized and stored in image memory. The 
second image is continually digitized while the user moves the film transport, and a 
video switcher alternately displays the stored and "live" images on a graphics screen 
at a frequency of about 4 Hz. There is an illusion of movement when the images 
are misaligned, and the movement is reduced as they are brought into alignment. 
When the motion is minimized between the two images or features of interest, the 
second ima.ge is stored. Next, serial sections two and three are aligned, followed 
by pairwise alignment of the remaining sections. Once the images are aligned, the 
features or boundaries of interest are traced manually using a bitpad. The traces 
can be displayed as a set of contours, as contours with the hidden lines removed, or 
tessellated to form a surface which can be displayed as a solid object (Harris and 
Stevens, 1989). 

Using motion to compare photographs of the same or similar objects is not a 
new idea. Astronomers have used blink comparators since early this century to study 
astronomical plates (Croswell, 1990). A blink comparator holds two plates of the 
same part of the sky, and alternately displays them to the user. Stationary objects 
such as stars remain fixed, but objects such as comets or planets appear to move. 

The problem of image alignment appears in many disciplines. Cartographers need 
to align aerial photographs, and in robot vision much effort is devoted to registration 
of stereo pairs and temporal image sequences (Horn, 1986). In both these disciplines, 
the images are different views of the same object, and in robot vision one can often 
assume that, at least locally, images are only misaligned translationally. The align- 
ment of neural sections, however, is a considerably more difficult problem because the 
EM images are misaligned both translationally and rotationally and there is gener- 
ally a large discrepancy between consecutive images. The latter is due both to the 
integration over the thickness of the section in the EM photographic process and to 
distortions of the tissue by its preparation and by the EM process. 

Giertsen et al. (1990) propose a method to automatically align electron micro- 
graphs. They manually trace the membrane and internal substructures of a pancreatic 
cell in serial sections and specify connectivity relations among successive contours. 
Using contour shape features (centroids and mean radii), they determine linear trans- 
formations between successive contours and refine the alignment using residuals be- 
tween the original and smoothed data. Aligned contours are tessellated and rendered 
as polygonal surfaces. 

Segmentation: Many techniques have been developed for 3D image segmenta- 
tion. One of the simplest techniques, intensity thresholding, has been used by (Drebin, 
Carpenter and Hanrahan, 1988) and (Hohne et al., 1989) and works relatively well 
for separating soft and hard tissue in MR and CT scans. Surface normals are needed 
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for high-quality rendering, and they may be computed using 3D generalizations of 
2D edge detectors; e.g., the Zucker-Hummel operator (Hohne et al., 1989). When the 
3D volume cannot be segmented by simply looking at single intensity values, a larger 
voxel neighborhood can be used, by applying a 3D generalization of the Marr-Hildreth 
operator (Hohne et al., 1988; Hohne et al., 1989). Other 3D generalizations of edge 
detectors have also been used for segmentation (Liu, 1977; Zucker and Hummel, 1981; 
Morgenthaler and Rosenfeld, 1981). 

Another technique for constructing polygonal representations of constant density 
surfaces from volumetric data is marching cubes (Lorensen and Cline, 1987). The 
data is assumed to be defined on a 3D lattice, and an iso-surface is approximated 
by finding all intersections between the iso-surface and the edges of the lattice. The 
technique dictates how to fill in the surface between the edges of a cube for each of the 
14 different types of intersections tha,t can occur between the surface and a cube in 
the lattice. The gradients are determined at each corner of the cube, and the surface 
is rendered using the gradients to estimate the surface normal. 

Unfortunately, because of the complexity of EM images of neuronal tissue (see Fig- 
ure 2), straightforward segmentation using any of the aforementioned techniques does 
not appear promising for 3D reconstruction of dendritic models. Therefore, we aim 
at introducing a higher level of automation into the section-by-section manual tracing 
methodology currently practiced by neuroscientists. A recently proposed model-based 
image feature localization and tracking technique known as snakes (Kass, Witkin and 
Terzopoulos, 1987) is well suited to this goal. This interactive technique is consistent 
with the manual tracing methods, but is considerably faster and more powerful. 

The ability of snakes to conform to complex biological shapes such as cells and 
to track their nonrigid deformations across image sequences makes them attractive 
tools for biomedical image analysis. In (Leymarie, 1990), snakes are used to track 
living cells moving on a planar surface. Assuming modest interframe motion, snakes 
can exploit fra,me-to-fra.me coherence to track the moving cell and also follow the 
deformations which occur as the cell moves. Ayache et al. (1989) use snakes to find 
edges in cross sections of MR data. The user draws an approximate contour around 
the region of interest and the snake deforms to fit the regions more accurately. Once 
the snake has reached an equilibrium, it is used as a starting point for the next cross 
section. A 3D model is built from the the resulting set of contours using Delauney 
triangulation (Boissonnat, 1988). Variations on snakes based on B-splines have been 
applied to the segmentation of 3D CT and MR data (Leitner et al., 1990). 

Visualization: Volume visualization is an area in computer graphics that has 
received a great deal of attention in recent years (Upson, 1989). Much work has been 
devoted to high-quality rendering of CT, PET, and MR data (Drebin, Carpenter 
and Hanra.han, 1988; Levoy, 1990) and seismic data (Sabella, 1988; Wolphe, Jr. and 
Liu, 1988). Specialized software and hardware has been developed to facilitate real- 
time manipulation of both medical data (Meagher, 1982; Meagher, 1984) and seismic 
data (Cha.kra.va.rty, Nichol a.nd Ono, 1986). More recently, general-purpose software 
for volume rendering has become available. Examples are ISG Technologies' ICAR 
System (Stevens and Trogadis, 1990), Stardent's AVS system (Upson et al., 1989) and 
Vitallmages' VoxelView system ( Vitallmages, Inc., 1990). In our work, we use the 
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VoxelView system on a Silicon Graphics workstation to visualize the 3D reconstructed 
dendrites. 

2 Reconstruction of Neuronal Dendrites 

2.1 Data Acquisition 

Using routine tissue processing, a slice (400/xm thick) of well-preserved hippocampus 
was obtained from the bra.in of a male rat, and the slice was embedded in epoxy 
resin. The slice was ftirther sectioned with an ultramicrotome, at an average section 
thickness of about 0.06/xm. (The dendritic segment used in this paper is dendrite 
number 24 of reference (Harris and Stevens, 1989)). Each section was photographed 
at 10,000 times magnification in a JEOL 100B transmission electron microscope and 
printed on 8x10 inch photographic paper (see Figure 2). 

The EM photomicrographs were digitized on an ECRM Autokon flat-bed laser 
scanner capable of digitizing reflection copy images at a wide range of resolutions 
(Ulichney, 1982). The scanner has a fixed spot size and the intensity can be quantized 
at one or eight bits/pixel. We digitized the images at a resolution of 2560x1983 pixels, 
approximately twice the sampling rate of the smallest features of interest, and with 
eight bits of intensity per pixel. The images were low-pass filtered and subsampled 
to a size of 640x496 pixels. 

2.2 Image Registration 

We chose a manual approach to image registration for two reasons. First, the au- 
tomatic alignment of successive EM images is a very difficult problem, because the 
images are displaced both translationally and rotationally and usually there is a large 
disparity between consecutive images. Second, even if an automated solution can be 
found, user intervention will be necessary when the tissue has been distorted during 
preparation; for example, when a section ha.s a fold. 

We have implemented an interactive digital blink comparator. One image is held 
stationary and the user translates and rotates the other image while the stationary 
and moving images are alternately shown on a graphics screen. The user can translate 
the image in the x- and y-directions and can rotate the image about its center by using 
a three-button mouse, each button controlling one type of motion. The user moves 
the image until the motion between the two images is minimized and the images are 
aligned. By using double buffering and by pre-computing x- and y-components of the 
composite transformation, we obtain comparisons at a frequency of about 1 Hz for 
a 640x496 pixel image on a Silicon Graphics 4D/220GTX using one of the R-3000 
processors. We found this frequency adequate to obtain good alignment, although a 
higher speed would be desirable. 

When all images are aligned by pairs, we find the composite transformation for 
each image relative to the first image in the EM series and resample the images using 
a spatially varying digital shift filter. This finite impulse response (FIR) filter is 
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designed with a conventional windowing technique (Oppenheim and Schafer, 1975). 
The resulting aligned images are input to the segmentation processes described in the 
next section. 

2.3 Extracting Models from Electron Micrographs 

The extraction of neuronal dendrites from a set of aligned EM images reduces to 
three subproblems: (i) the localization of dendritic profiles in digital micrographs, 
(ii) the segmentation of the interiors of dendrites bounded by profiles, and (iii) the 
identification of profiles of the same dendrite across serial micrographs. The density 
and geometric complexity of neuronal features in micrographs make the first and third 
subproblems especially difficult to automate fully. 

We take a semi-automatic approach which exploits recently developed physically- 
based vision techniques for interactively localizing and tracking extended features in 
images. We employ a variant of snakes, the interactive deformable contour models 
introduced in (Kass, Witkin and Terzopoulos, 1987). Snakes provide significant as- 
sistance to the user in accurately locating the membranes that bound the dendrites 
in EM images. Using a mouse, the user quickly traces a contour which approximates 
the dendrite boundary, then starts a dynamic simulation that enables the contour to 
locate and conform to the true membrane boundary. Where necessary, the user may 
guide the contour by applying to it simulated forces using the mouse. Through mini- 
mal user intervention, snakes quickly produce accurate dendritic profiles in the form 
of complete, closed contours that facilitate the segmentation of dendritic interiors. 
Finally, with some guidance, snakes are able to exploit the coherence between serial 
micrographs to quickly extract a sequence of profiles of the same dendrite. 

2.3.1 Deformable Contour Models 

A snake can be thought of as a dynamic deformable contour in the x-y image plane. 
We define a discrete deformable contour as a set of n nodes indexed by i = 1, . . . ,n, 
with time varying positions x,(t) = 3/t(01'- The behavior of an interactive 

deformable contour is governed by the first-order dynamic system of equations 

7 ^i +a .+0. = fi; i = l,..., n , (1) 

where 7 is a velocity-dependent damping constant, cti(t) are "compression" forces 
which make the snake act like a series of unilateral springs that resist compression, 
fii(t) are "rigidity" forces which make the snake act like a thin wire that resists 
bending, and f t (i ) are forces in the ima.ge plane applied to the contour. 

Let be the given reference length of the spring connecting node i to node i -f 1 
and let r,(i) = x, +1 — x, be the separation of the nodes. Given the deformation 
e,(Z) = ||r,|| — we define 



_ cqej _ a t -iej_i 
N| r< Hr^lf*- 1 - 



(2) 
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To obtain contours that can stretch arbitrarily but resist shrinking past a prespecified 
amount, we set 

. fa if e { < 0, , . 

atW = \ 0 otherwise, (3) 

so that each spring resists compression with constant a only when its actual length 
|| is less than To give the contours some rigidity, we introduce the variables 6, 
and define rigidity forces 

ft = f> I+ i(x l+2 - 2x l+1 + x t ) - 2bi(x i+1 - 2xi + x,_ a ) 

+ &i_i(xi - 2xi_i + Xi_ 2 ). (4) 

Note that in the absence of external forces, if the nodes are separated more than li, 
are equally spaced, and lie on a straight line, and /3 i vanish and the contour will 
be at equilibrium. Compression and rigidity are locally adjustable through the 
and 6, variables. In particular, by setting a l = b t — 0, we are able to break a long 
deformable contour into several shorter contours on an image. 

To simulate the deformable contour we integrate the system of ordinary differential 
equations (1) forward through time using a semi-implicit Euler procedure (Press et al., 

1986) . Applying the forward finite difference approximation dxi/dt « (x- +At — x*)/ At 
to (1) and collecting linear terms in the x t on the left yields the pentadiagonal system 
of algebraic equations 

^x^+Zr^-x'^ + f? (5) 

for the subsequent node positions x' + Af in terms of the current positions x'. Since the 
system lias a constant coefficient matrix, we factorize it only once at the beginning of 
the deformable contour simulation using a direct LDU factorization method and then 
efficiently resolve with different right-hand sides at each time step (see (Terzopoulos, 

1987) for details). 

After each simulation time step we draw lines between the new node positions 
x' +At to display the deformable contour as a dynamic curve in the image plane. 



2.3.2 Image Segmentation using Deformable Contours 

The deformable contour is responsive to an image force field which influences the 
contour's shape and motion. It is convenient to express the force field as the gradient 
of a potential function P[ s (x,y) computed from the image I,(x,y) of EM section 5: 

f t = VP 7 ,(x t ), (6) 

where V ~ [dj dr., d/dy}'. By simulating (1) with (6), the ravines (extended local min- 
ima) of P Is act as attractors to deformable contours. The contours "slide downhill" 
and stabilize at the bottoms of the nearest ravines. 

In the present application, we are interested in the localization of cell membranes, 
which appear dark in positive micrographs. We therefore convert I,(x,y) into a 2D 
potential function whose ravines coincide with dark cell membranes: 



PlA x ,v) = G a * I,{x,y), 



(7) 
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Figure 3: Cell segmentation using a. deformable contour (see text), (a) Initial sketched 
contour, (b) Initial equilibrium position, (c)-(d) Manipulating the contour with 
interactive springs (green lines) and constraints (blue lines), (e) Final profile, (f) 
Segmented cell. 
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where G^* denotes convolution with a 2D Gaussian smoothing filter of width a. The 
filter broadens the ravines of Pj t so that they attract the contours from some distance 
away. 

In practice, /, is not a continuous function, but a digital image. Therefore, we first 
convolve the image with a discrete smoothing kernel, then compute (6) by bilinearly 
interpolating the smoothed image gradients evaluated at the four pixels surrounding 
x^. The user can set the degree of smoothing a and select the display of image I, or 
potential functions Pj s through a menu-driven interface. 

The user initializes a closed deformable contour by quickly sketching with a mouse 
an approximate trace around the dark membrane of a dendrite of interest. Figure 
3(a) shows an initial deformable contour sketched near a cell membrane (nodes are 
created automatically so that they are spaced about one pixel apart, and an additional 
spring is inserted between the first and last nodes to close the contour). The user 
then initiates the snake simulation (5). In a few simulation time steps the deformable 
contour equilibrates at the bottom of the nearest ravine in Pj t (Figure 3(b)). By 
interacting with the contour (see below), the user can help it quickly localize the 
membrane ravine conform to its shape to produce an accurate profile of the dendrite 
(Figure 3(e)). 

Because the dendritic profile is a. closed continuous contour, it is easy to segment 
the interior of the cell from the rest of the image. We accomplish the segmentation by 
applying a standard region-fill algorithm (Foley et al., 1990) which starts from a seed 
point inside the profile and sequentially accesses the pixels of I„ that are bounded 
by the profile. Figure 3(f) shows the cell segmented from the dendritic profile of 
Figure 3(e). 



2.3.3 User and Constraint Forces 

Often the user will sketch an initial trace which deviates too much from the membrane 
ravine to descend into the ravine properly. This is the case for the initial contour 
in Figure 3(a) which reaches the equilibrium configuration shown in Figure 3(b). As 
can be seen, the deformable contour may fall prey to nearby dark features inside the 
cell (or to nearby features of neighboring cells) which act as false attractors. In such 
a case, the user may apply interactive simulated forces f t m (£) by using the mouse to 
guide deformable contour towards the ravine of interest as it is stabilizing (see (Kass, 
Witkin and Terzopoulos, 1987) for details about user forces). A useful force is the 
interactive spring 

fm _ f x t ~ m (0 if Il x i ~ m (OII 1S minimal for node i, 
1 1 0 otherwise. 

which pulls the nearest node towards the time-varying mouse position m(i) in the 
image plane. Figure 3(c) shows the effect of a user stretching the contour towards the 
right with an attached interactive spring (green line) from the mouse position (blue 
circle). 

To localize a profile accurately, the user may want to constrain points on a de- 
formable contour by attaching them with springs to selected anchor points on the 
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image. Such constraints prevent the deformable contour from straying far from these 
points, regardless of the image forces and the user's other mouse manipulations. The 
mechanism for adding constraints is simply to fix ni(i) = in the spring force (8) to 
create an anchor point a*, in the image. The constraining spring then applies a force 
ff fc . Figure 3(d) illustrates a constraint spring (blue line) which pulls the contour 
back towards an anchor point on the cell membrane as the user tugs on the contour 
with an interactive spring (green line). Note the two constraints (blue dots) in the 
final profile contour in Figure 3(e). 

Combining the three types of forces, we have 

f. = C/VP^X.) + Cn.f," 1 + c a £ f°\ (9) 

fc 

where Ylk ls a summation over all the anchor constraints in force and where cj, c m , 
and c a are the strength factors of the image forces, user spring forces, and anchor 
spring forces. 

2.3.4 Exploiting Coherence Across Serial Sections 

Our interactive technique for extracting cell profiles from EM images benefits from 
the fact that snakes can exploit the coherence of profile positions and shapes across 
adjacent images. Often, the user need not reinitialize the deformable contours when 
progressing from image to image to extract adjacent profiles of a dendrite. 

Once the deforma.ble contours equilibrate into the membrane ravines in Pi t , we 
replace this potential function with the potential function Pi, ±1 of an adjacent member 
of the image sequence. Continuing from their previous equilibrium positions, the 
contours automatically slide downhill to regain their equilibria in the new ravines, 
quickly localizing the new positions of the cell membranes in the adjacent image and 
conforming to their shapes. 

This simple mechanism for exploiting image-to-image coherence works so long as 
the perturbation is small enough to maintain the deformable contours within the 
membrane ravines as we switch to adjacent images. Should part of a contour escape 
the ravine, however, the rest of the contour will usually pull it back into place due 
to the rigidity of the model. Nonetheless, deformable contour reinitializations are 
normally required when portions of the dendritic spines are no longer connected to 
the parent dendrite. 

3 Visualizing Neuronal Dendrites 

During image segmentation, the user can visualize what takes place in two graphics 
windows (Figure 4). The left window shows the current EM image overlayed with 
the deformable contour simulation. The right window displays a stack of dendritic 
profiles and their interiors. The user can also see the current contour with the (partial) 
model, thereby monitoring progress. The stacked set of contours and interiors can 
be rotated and viewed from different vantage points. When all the dendritic profiles 
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Figure 4: To the left, the current EM image with a deformable contour (snake); to the 
right, a stack of dendritic profiles and their interiors. The current snake is displayed 
in red. 



have been found, we use the segmented dendritic interior to build a volumetric voxel 
model. We apply volume rendering techniques to visualize the volumetric model. 

Volume rendering refers to the direct rendering of scalar data sampled in three 
dimensions. These techniques differ from traditional computer graphics techniques 
in that explicit surfaces need not be extracted from the data before display. Rather, 
the entire 3D volume of data is used for display. Yet, by displaying only the portions 
of the volume that have a given density or a high gradient, features and surfaces may 
be elicited without explicit representation (Foley et al., 1990). 

We use the VoxelView system (Vitallmages, Inc., 1990) to reconstruct a 3D vol- 
umetric model of a dendrite from the dendritic interiors that have been segmented 
from the serial sections. The segmented dendrite in each image is enclosed by a rect- 
angular array of black pixels. The resulting arrays, or images, are stacked in order. 
The sizes of the rectangular arrays are chosen so that the stack yields a rectangular 
parallelepiped. Since the sampling rate is less in the stacking direction (z-direction) 
than in the x- and y-directions of the images, the VoxelView system linearly inter- 
polates an additional number of sections. In the examples shown below, we have 
reconstructed 41 sections of a. dendrite, and interpolated three sections between each 
pair of original sections to get approximately the correct proportions along the z-axis 
as we view all the sections. 

By rapidly moving from one image to the next, we generate an interactive "movie" 
that enables us to follow certain features of interest through the dendrite. We can 
also slice the stack of images along planes perpendicular to the stacking direction, 
and along any arbitrary plane. Because of the tissue cutting direction, the dendrite 
is positioned obliquely in the image volume; therefore, we must cut the image volume 
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Figure 5: Oblique slice through model of a dendrite reconstructed from 41 serial 
sections. The mitochondrion can be seen extending through the length of the dendrite 
and some smooth endoplasmic reticulum is visible in the large spine. 




Figure 6: Shaded model of the reconstructed dendrite. 
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obliquely to slice the dendrite lengthwise. One such oblique slice is shown in Figure 5, 
where we can see the. shape and extent of the mitochondrion through the dendrite, 
and also some smooth endoplasmic reticulum. 

By default, the volume is rendered without any shading; however, the user can 
specify the position of a light source and obtain a shaded view of the model. In 
Figure 6, we have rendered the dendrite model with shading. The shading helps 
accentuate the 3D shape of the dendrite. Other features of the VoxelView system that 
we find useful are the ability to tumble the model interactively and to make portions 
of the model transparent by adjusting the opacity values of the pixels (Vitallmages, 
Inc., 1990). 

By using a volumetric representation of the dendritic model, we can represent the 
3D shape of the model with accuracy limited only by the original sampling of the EM 
images. Furthermore, we can visualize the cytoskeleton and the organelles interior to 
the dendrite. 

4 Summary and Future Research 

We have described a prototype system for the reconstruction and analysis of neu- 
ronal dendrites. Our goal is to reduce the effort required to reconstruct and analyze 
a complete dendrite from a few months to a few days. We are approaching this goal 
by exploiting three recently developed techniques for volume reconstruction: a digital 
blink comparator for EM section registration, snakes, or active energy-minimizing 
contours, for dendrite segmentation, and volume rendering to visualize both the over- 
all morphology of 3D dendrites and their cytoskeleton and internal organelles. 

Much work remains to be done to improve the reconstruction process. The digital 
blink comparator opens a way to use direct digitization from electron microscopes 
for serial microscopy. This would eliminate the need for rephotographing and digi- 
tizing EM photomicrographs, thus reducing the reconstruction time and eliminating 
distortions and quantization errors introduced by these processes. Direct digitization 
from an electron microscope has been used for single section studies, but has until 
now been impossible to use for serial microscopy, which requires section alignment 
(Stevens and Trogadis, 1984). 

We need to provide (at least) a semi-automatic approach to image registration, 
with the user intervening only for optional fine-tuning. Currently, during manual 
alignment, we resample the image using nearest pixel sampling. It would be desirable, 
however, to allow subpixel alignment and resampling by using a spatially varying shift 
filter, but this is computationally prohibitive with our current equipment. Despite the 
sampling limitations, we have found the resulting alignments to be quite satisfactory. 

We need to improve upon the snakes' behavior at spine branching points, to reduce 
the amount of user assistance required when dendritic spines are no longer connected 
to the parent dendrite. 

We need to improve upon interslice interpolation to allow fractional section inter- 
polation; that is, proper resampling of the volume to get accurate proportions in the 
x-, y-, and z-directions. Similarly, we need to provide better inter-pixel interpolation 
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at volume slicing and rendering in order to reduce aliasing without compromising 
image accuracy. 

We have not yet begun to tackle anatomical analysis of the dendrites. We ex- 
pect, however, that through (semi-)automated approaches for dendrite decomposi- 
tion, anatomical measurements, and statistical analysis of these measurements, we 
can achieve reductions in analysis times similar to those that we are beginning to 
realize in the reconstruction phase. 

When we have reached our goal, the time required to reconstruct and analyze 
neurons or parts of neurons will be reduced from a few months to a few days. It will 
then be possible to obtain a sufficiently large number of reconstructions to evaluate 
quantitatively the functional consequences that alterations in neuronal morphology 
have for both the normal and diseased brain. 
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